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Abstract 


This study expands upon previous investigations regarding factors that affect individual ability to 
provide accurate responses via self-report measures. Specifically, the present study investigated the 
relationship between cognitive ability and response inaccuracy (aberrance), the hypothesis being that 
response inaccuracy on personality self-report was affected by cognitive ability. Cognitive abilities 
were measured using the Intelligence Structure Test; IST-70 and self-report accuracy was measured 
using the Coopersmith Self-Esteem Inventory (CSEI). Both instruments were administered to 11th and 
12nd-grade students from 14 schools in Indonesia (N = 1,674). The W-statistic, a person-fit statistic in 
Rasch modeling, was employed to identify inaccurate response patterns on the CSEI. Logistic 
regression showed significant effects for cognitive and verbal abilities on individual response accuracy 
on the CSEI: individuals with higher cognitive abilities were more likely to produce accurate responses 
than those with lower cognitive abilities. In addition, the consistency of individual response accuracy 
ranged from low to high depending the characteristics of the specific item. Respondents demonstrated 
high consistency in their ability to give accurate responses within the same measurement for similar 
items. 
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1. Introduction 


Personality measures are increasingly being used as instruments to collect data in 
the social and behavioral sciences. Most personality measures are performed using 
self-report or questionnaire-type assessments. Since the accuracy and validity of self- 
report depends on how individuals perceive themselves, this technique is vulnerable 
to contamination via inaccurate (aberrant) responses. Various permanent factors also 
affect individual self-report of the attribute being assessed (Urbina, 2004). On the 
psychological level, those factors include cognitive and personal characteristics, such 
as level of motivation, level of test anxiety, and tendency to guess (Hambleton, 
Swaminathan, & Rogers, 1991). Measurement error on self-assessments has also been 
attributed to maladjustment, low socioeconomic status, high individualism, marginal 
reading proficiency, differences in cultural background, low levels of cooperation, and 
inattention (Domino & Domino, 2006). However, little attention has been paid to 
these individual-level sources of measurement, despite strong evidence that certain 
individuals provide more accurate responses than others to any given assessment item 
(Castro, 2013). The present study therefore focused on the role of individuals in 
inaccurate responses on a Self-report assessment, which led to low poor item fit 
parameters as indicated by person fit. 

When individuals respond to a survey question, it requires substantial cognitive 
effort to provide an appropriate answer, expressed in a way that fits the measurement 
model of the employed measure. According to Schwarz (2007), individuals need to 
interpret the item to understand what is meant and to determine what information they 
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should provide. If the purpose of the instrument is to measure an attitude, respondents 
retrieve a previously formed attitude from memory. They also form a judgment at the 
time of assessment, based on whatever relevant information is accessible at that point 
in time. Krosnick (1991) propose three factors that contribute to the likelihood of 
adopting a response strategy: task difficulty, cognitive ability, and motivation. 
Cognitive ability refers to the respondent’s capabilities with respect to executing the 
cognitive processes required to formulate a response to the item in question. This 
factor comprises individual differences in cognitive skills, level of education, and 
knowledge of the survey topic. A number of authors have proposed this cognitive 
factor to be influential on other pertinent factors (e.g. Castro, 2013; Dykema, 
Lepkowski, & Blixt, 2012). For example, cognitive components might influence a 
respondent’s anxiety, since the respondent’s inability to concentrate fully on the task 
(due to cognitive limitations) can produce anxiety. Cognitive attributes can also affect 
whether a respondent’s attention to the survey task relates to that respondent’s level of 
motivation (Wine, 1971). 

Cognitive abilities are an important factor in assessing individuals’ behavior and 
performance. For example, cognitive abilities influence measurement precision 
related to the respondent’s ability to read and interpret questionnaire instructions and 
items (Stone, Stone, & Gueutal, 1990). Lower cognitive ability can lead to 
inappropriate responses, decrease measure precision, increase the number of items left 
unanswered, and trigger a submissive attitude. (Viswanathan, 2005). The present 
study assumes that accurately responding to a questionnaire requires substantial 
cognitive effort (Krosnick, 1991). 

Cognitive ability implies a capacity to process complex information (Halford & 
Andrews, 2011). According to Peacock, Ervin, and Daly (2009), cognitive ability is 
defined as any ability concerning a class of tasks that requires the correct or 
appropriate processing of mental information. Cognitive ability is divided into four 
main dimensions: verbal ability, reasoning, perceptual speed, and memory (i.e. 
Woodcock, 2002). Verbal ability requires an understanding of the concepts and logic 
underlying item wording, allowing respondents to both decode the stimulus and 
encode the response. Reasoning is generally defined as solving a problem using 
logical and practical thinking, and is required to interpret and comprehend item 
wording. This dimension also depends on whether the required pieces of information 
are readily available (Schweizer & Koch, 2003). Perceptual speed refers to the speed 
with which mental comparisons are made, and manipulate information in short-term 
memory stores (Boyle, Wilson, Schneider, Bienias, & Bennett, 2008). Regarding to 
answering item on the scale, this dimension explain respondents’ ability to interpret 
questions and make them relevant to their own experiences with regard to the speed. 
Schweizer and Koch (2003) have examined perceptual processes as essential parts of 
cognitive ability. Exploring different stimulus complexity, response modes, level of 
encoding, and attention, these authors found that the largest source of variance for 
perceptual processes and cognitive abilities was attention. 


1.2. Cognitive Ability and Response Accuracy 


Cognitive ability is important to providing accurate responses on assessments. 
Truxillo, Seitz, and Bauer (2008) investigated the role of cognitive ability in 
moderating the relationship between actual and perceived test performance. They 
found a high correlation between actual and perceived performance for participants 
with high cognitive ability, but not for those low with cognitive ability. Their finding 
was consistent with Stone et al.’s (1990) finding that the reliability of measures was 
significantly greater when the measure was administered to respondents with high 
cognitive ability. Another study also suggested that individuals with higher cognitive 
ability give more accurate responses on scale measures (Hauenstein & Alexander, 
1991; Schwarz, 2007). Together, these authors provide strong evidence that the 
cognitive ability of respondents influences the quality of measurements. 

Cognitive abilities are involved in many theories that seek to describe how accurate 
responses are produced. According to Cannell’s process theory, an accurate response 
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is the product of five sets of processes: (a) comprehension of the question, (b) 
cognitive processing, (c) evaluation of the accuracy of the response, (d) evaluation of 
the response based on goals other than accuracy, and (e) accurate responding 
(Cannell, Miller, & Oksenberg, 1981). Tourangeau’s response process theory involves 
four cognitive ability components, each of which demands cognitive resources: 
comprehension, retrieval, judgment, and response; Tourangeau’s theory also involves 
13 relevant cognitive processes such as assessments concerning the information 
sought, retrieval of relevant memories, integration, and response formulation 
(Tourangeau, Rips, & Rasinski, 2000). Krosnick’s satisficing theory (Krosnick, 1999) 
also seeks to explain how individual cognitive ability affects accurate responses on 
questionnaires. Individuals may process item questions to differing depths and may 
not carry out each element of the process with the same level of attention. Thus, it 
should not be assumed that respondents who are inattentive while reading an item 
question will have poor information retrieval. According to this theory, the key point 
is that individual may carry out each cognitive operation carefully. 

Individuals may also provide incorrect responses due to satisficing (Krosnick, 
1991). According to author, satisficing is a behavior that generating a merely 
satisfactory response instead of attempting to generate an accurate response. 
According to satisficing theory, there are three factors that affect the question 
answering process: motivation of the respondent to perform the task, the difficulty of 
the task, and respondent’s cognitive ability to perform the task (Borgers & Hox, 
2000). More specific to cognitive ability, Krosnick (1991) reports that three aspects of 
individual cognitive ability relate to satisficing. First, retrieving information and 
making judgments should be easier for individuals who are adept at performing 
complex mental operations. Second, individual ability is related to the amount of 
practice that an individual has had at thinking about the topic of a particular question, 
with more experienced individuals being better able to answer relevant questions. 
Third, individual ability is related to the degree to which individuals have pre-existing 
attitudes on the issue in item: some people have unambiguous evaluations of objects 
stored in their memories, which are easily accessible and may therefore be called to 
mind with little effort. All of these three satisficing aspects are associated with 
cognitive abilities because optimally response to item question or statement require 
substantial cognitive effort, however some respondents simply provide a satisfactory 
answer instead (Krosnick, 1991). Several authors support this notion (e.g. Kaminska, 
McCutcheon, & Billiet, 2010; Lenzner, Kaczmirek, & Lenzner, 2010), they found that 
satisficing and reluctance to participate in surveys were related to cognitive ability. 

The cognitive ability of memory retrieval also plays an important role in producing 
accurate responses. Items on self-report assessments mostly ask about information 
related to individual past experiences. To give an accurate response, respondents are 
expected to be capable of retrieving their memories related to the question. Thus, 
good memory function is required to respond accurately to items that ask about 
behaviors within a certain time frame. When responding to items on a questionnaire, 
several processes are involved in memory retrieval, including adopting a retrieval 
strategy, generating specific retrieval cues to trigger recall, remembering individual 
memories, and filling in partial memories through inference (Collins, 2003). 

An accurate response is defined as a valid response that accurately describes the 
condition of the individual. This definition is based on literature on survey research 
(Brace, 2004; Schuman, 1980). To determine whether an individual’s response is 
accurate, external information is usually required to verify individual reports. 
However, there are some procedures that can be performed to determine the accuracy 
of individual responses, such as examining the consistency of responses, since 
accurate responses tend to be more consistent than inaccurate responses. This 
consistency can be determined by comparing responses for items that measure the 
same attributes, which should therefore be answered consistently. Another approach 
to determining the accuracy of responses is to examine the response pattern. 
According to this approach, an accurate response is indicated by a response pattern 
that corresponds to the psychometric properties of the item (e.g. item location 
parameter). As literature on Rasch model (Bond & Fox, 2001) , each item on the scale 
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might has different location parameter, which can represent the level of intensity on 
the latent trait continuum that an individual would possess in order to have a 50% 
probability of endorsing the item. For example, a Rasch analysis of the Item 5 resulted 
in item location estimates of 1.5. Thus, individual with trait level (0; theta) equal to 
1.5 have 50% to endorse the item. In addition, individuals with trait level below 1.5 
have below 50% probability of endorsing item and individuals with trait level above 
1.5 have above 50% probability of endorsing item. An accurate response is indicated 
by individual’s response that follows that notion. Hence, an accurate response refers 
to responses that match to the individual’s latent trait with regard to item location. 

By comparing items that have differences in difficulty level, the accuracy of a 
response can be detected. In Rasch modeling, difficulty level is denoted by the item 
location parameter (Bond & Fox, 2001). Based on the item location parameter, an 
accurate response is signified by a response that reflects a matching between item 
location parameter and individual trait level. Conversely, an inaccurate response, 
subsequently referred to as an aberrant response, is a response that does not match to 
item location with regard to his or her level of ability. For example, the more often 
that a participant agrees with easy items but disagrees with difficult items that 
measures the same attributes, the higher his or her index of aberrancy. 


1.3. Person-Fit Statistics as Aberrant Response Detector 


Person-fit statistics identify respondents with unusual response patterns by 
comparing their responses to the modeled expectations. Person-fit measurement refers 
to statistical methods used to detect improbable item-score patterns. These statistics 
also determine the degree of consistency between the pattern of responses across the 
entire personality measure and estimation of the latent trait. The relationship between 
person-fit statistics and the test-taker’s actual characteristics can be examined to 
generate hypotheses about what characteristics are common in persons with aberrant 
responses to testing. 

Reise and Flannery (1996) described several key concerns regarding person-fit 
assessment and scalability indexes. The three major goals are to identify (a) specific 
aberrant test behaviors that presumably result from specific concerns relating to test- 
taking, (b) the nature of the test or test administration, and (c) psychological processes 
underlying aberrant responses. Reise and Flannery’s study assessed the third concern, 
exploring whether cognitive ability influenced respondents’ likelihood of giving 
aberrant responses. 

A number of studies comparing the use of person-fit statistics in detecting aberrant 
responses found that the following person-fit statistics performed the best: the 
standardized log-likelihood person-fit statistic (Li & Olejnik, 1997), U statistics, and 
W statistics (Linacre, 1997). U and W statistics can be used to identify departures 
from logical response patterns, and were developed specifically for the Rasch model. 
Derived from the unweighted mean square, the U statistic averages the squared 
response residuals over several items. The W statistics is derived from the weighted 
mean square and is estimated using the average of the item response residuals, and 
weighted by the sum of the variances (Karabatsos, 2003). Although there are various 
types of coefficients to indicate person-fit, W statistics are the most widely used. 


2. Methodology 
2.1. Objective 
The present study investigated the relationship between cognitive ability and 


response inaccuracy (aberrance response). The hypothesis of this study is a response 
inaccuracy on personality self-report was affected by cognitive ability. 
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2.2. Participants 


Participants of this study were high school students in 11th and 12th grade. This 
sample was selected taking into account age, gender, and school location to best 
match the student population in Indonesia. First, 851 students participated in an item 
calibration study (field testing) to estimate the item parameter of the self-esteem 
measure. Then 1,674 students participated in hypothesis testing about the effects of 
cognitive ability on person-fit. Participants in the hypothesis testing were 36.8 % male 
and 63.2 % female, with a mean age of 17.3 years. 


2.3. Instrument 


Cognitive Ability. Cognitive ability was measured using the Intelligenz Struktur 
Test (Intelligence Structure Test; IST-70). The IST-70 was first published by Rudolf 
Amthauer in 1953. A revised version was released in 1970, which had gone through 
many editions and translations (Amthauer, 1973). This paper-and-pencil test contains 
nine sub-scales: sentence completion (SE), verbal classification (WA), verbal 
analogies (AN), verbal concept formation (GE), numerical tasks (RA), number series 
(ZR), figure matching (FA), spatial orientation (WU), and memory (ME). These 
subtests were grouped into four ability categories: (a) verbal reasoning (sum of SE, 
WA, AN and GE subtest scores), (b) quantitative reasoning (sum of RA and ZR 
subtest scores), (c) figural reasoning (sum of FA and WU subtest scores) and (d) 
memory (ME subtest score only). The split-half reliability of the test has been found 
to be 0.97 for the entire test and ranging from 0.86 to 0.96 for individual subtests. 
Reliability with a parallel form was found to be 0.95 (Mehrmann, 2004). 

Self-esteem. Self-esteem was measured using the Coopersmith Self-Esteem 
Inventory (CSEI; Coopersmith, 1981). The CSET is a self-report test consisting of 25 
statements measuring four domains of self-esteem: home—family, peers—friends, 
school—academic, and general self-esteem. Items (e.g., “I usually imagine as another’; 
“At times, I think I am no good at all”) are rated on a four-point Likert-type scale 
from 1 = “totally disagree” to 4 = “totally agree.” Coopersmith (1981) presents strong 
support for the reliability and validity of the measure. Reliability measure are 
generally good, varying in different studies between .80 and .92. The Indonesian 
version of the CSEI showed similar reliability coefficients (Hadjam, Martaniah, 
Prawitasari, & Masrun, 2004). For the total score the reliability coefficient was .82, 
and the coefficients for each subscale ranged from .78, for the general self-esteem 
score, to .58 for the social—peers self-esteem score. 


2.4. Procedure 


Data were collected in the classroom after all academic subjects were finished for 
the day. Before the tests were administered, the students were encouraged to see this 
testing session as a practice for later college admission testing. Students were asked to 
create and indicate a personal identification number on the answer sheets to ensure 
anonymity of responses. Participation in this study was optional, and students were 
invited to participate in the study during the regular school day. To increase students’ 
motivation to participate and to encourage them to take the testing seriously, they 
were told that the tests would be useful in predicting their future academic success. 


2.5. Data analysis 


Person-Misfit Detection. Fit statistics between items and measurement models were 
assessed using the W coefficient (mean-squares statistic) under the Rasch Model. In 
general, mean squares close to 1.0 suggest little distortion of the measurement system. 
Mean-squares value can be divided into four categories: (a) above 2.0, indicating that 
the individual’s responses distort or degrade the measurement system, (b) above 1.5 to 
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2.0, indicating that the individual’s responses are not helpful for constructing a 
measurement system, but do not degrade the measurement system, (c) 0.5 to 1.5, 
indicating that the individual’s responses are helpful for constructing a measurement 
system, and (d) below 0.5, indicating that the individual’s responses are less 
productive for constructing a measurement system; responses in this category do not 
degrading the measurement system, but may produce misleadingly good reliabilities 
and divisions. To simplify the analysis for hypothesis testing, the mean-square value 
is divided into two categories: a code value of “1” is assigned to accurate responses 
(i.e., mean-square values above 2.00 and below 0.50), and a code value of “O” is 
assigned to aberrant responses (i.e., mean square values of 0.50 to 2.00). Each 
individual produces a mean-square statistic and a Rasch score; the analysis therefore 
obtains the same number of person-fit statistics as the number of individuals being 
analyzed. In the present study, these values were these correlated with the cognitive 
ability of each individual. The WINMIRA procedure for estimating person-fit was 
employed in the present study (von Davier, 2001). 

Logistic Regression. The effect of individuals’ cognitive ability on person-fit was 
analyzed using logistic regression since the dependent variable (i.e., person-fit) is 
binary. In this technique, the logit is the natural logarithm of the odds or the likelihood 
ratio that the dependent variable is “1” (accurate response) as opposed to “0” (aberrant 
response). The analysis was performed twice: the first analysis included a predictor of 
cognitive ability, which was inferred from the IST total score; the second analysis 
used the IST sub-scores for the four specific cognitive abilities (verbal reasoning, 
quantitative reasoning, spatial reasoning, and memory). The second analysis entailed 
logistic regression using the forward method with Wald-¥2 test results as the criteria 
to exclude non-significant predictors. The final models thus contained only predictors 
that had a significant effect on the person-fit. Missing values were deleted using 
pairwise deletion. A p-value below .05 was used as the cut-off value to reject the null 
hypothesis. To examine the consistency of individuals in reporting accurate or 
aberrant responses, person-fit within individuals was correlated using Phi coefficients, 
since the person-fit is binary. 


3. Results 


The scores for general cognitive ability as measured by the IST-70 (M = 99.551; 
SD = 11.864) and for self-esteem as measured by the CSEI (M = 69.106; SD = 8.864) 
both followed normal distributions. A low correlation (r = 0.114; p<0.001) was found 
between cognitive ability and self-esteem. 


3.1. Item Calibration and Person-Misfit Detection 


In order to assess whether items in the measure fit Rasch scalability, Q-indices for 
all items were computed (Rost & von Davier, 1994). All self-esteem items showed a 
good fit to the partial credit model (PCM) since no items yielded Q-values above 
0.30, or were statistically significant. Table 1 shows item location (1.e., Rasch 
measure) on the logit scale and the number of individuals whose responses were 
identified as aberrant. Item location varied, with values ranged from a negative value 
(easy to be endorsed) to positive (difficult to be endorsed). For example, item location 
on the negative factors of self-esteem ranged from -1112 to 1616 (on the logit scale). 
The results of Rasch analysis using item parameters obtained from the calibration 
study on the first sample, which served as an anchor, show that 62-67 % of 
individuals in the second sample (hypothesis testing) gave aberrant responses. 

Chi-square testing indicated a statistically significant relationship between person- 
fit and school location (more popular school vs. less popular school) for positive self- 
esteem (y2 = 9.305; p<0.01) and family (y 2= 6.617; p<0.05), but not for negative 
self-esteem (y2 = 3.438; p>0.05). Individuals from the more popular school produced 
more responses that fit the model than individuals from the less popular school. There 
were no person-fit differences between males and females for positive self-esteem (y2 
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= .254; p>0.05) or family (y2 = .938; p>0.05), but there was for negative self-esteem 
(¥2 =5.002; p<0.05). 


Table 1. Item parameters of each self-esteem factor and number of persons identified as giving 








aberrant responses 
Self-esteem Factor No. Items Item Location Misfita 
Positive Self-esteem 8 -0.592-0.620 62 % 
Family 6 -0.433-0.465 63 % 
Negative Self-esteem 11 -1.112-1.616 67 % 





Note. A Misfit indicates percentage of individuals identified as giving aberrant responses. 
Effect of Cognitive Ability on Person-Fit 


Logistic regression analyses were conducted to examine whether cognitive abilities 
uniquely predicted the appropriateness of individual responses, as indicated by the 
person-fit coefficient. Logistic regression analyses showed that cognitive ability was 
significantly related to person-fit (Table 2). These findings were consistently found 
for each factor of self-esteem. The value of the odds ratios (OR) ranged from 1.016 to 
1.020, and the Wald-y2 test values ranged from 12.423 to 21.762 (p<0.001). 
Furthermore, the Hosmer-Lemeshow goodness-of-fit test had a non-significant 
(p>0.5) value, suggesting that the final model fit the data appropriately. Since high 
coding of person-fit values indicate fitting responses, and low coding values indicate 
poorly fitting (aberrant) responses, positive values for the B parameter indicate a 
positive effect of cognitive ability on response fit. 


Table 2. Logistic Regression Models Evaluating the Effect of Cognitive Ability on Person-fit 





Wald Odds  Nagelkerke 


Variable B SE x2 Ratio (R-squared) 





Factor 1 (Positive Self-Esteem) 








Cognitive Ability 020 .004 = =20.864* 1.020 017 
Constant -2.468 435 32.176* —.085 

Factor 2 (Family) 
Cognitive Ability .020 004 = =21.762* 1.020 018 
Constant -2.580 439 34.570* 076 

Factor 3 (Negative Self-Esteem) 
Cognitive Ability 016 004 =12.423* 1.016 .016 
Constant -2.286 448 26.042* —.102 





Note. B indicate coefficient of logistic regression. SE indicates standard error of estimate. * p<0.001 


More specifically, logistic regression was conducted on the four IST subtests 
(verbal reasoning, quantitative reasoning, spatial reasoning, and memory). The 
forward method was employed so that only the significant predictor was included in 
the final model (Table 3). Analyses showed that only verbal reasoning significantly 
predicted whether individuals would give accurate responses to each factor of self- 
esteem. Odds ratios ranged from 1.005-1.007 and the Wald-y2 test values ranged 
from 12.681-—26.642 (p<0.001). The Hosmer-Lemeshow goodness-of-fit test was also 
non-significant (p>0.5) for all models, and B values were positive, implying that 
higher verbal reasoning scores affected the appropriateness of individuals’ responses. 


Consistency of Person-Misfit 


To identify whether or not the individuals gave consistently aberrant or accurate 
responses, the correlation between individual person-fit statistics was computed. 
Higher correlations indicate that individuals respond to each self-esteem factor in the 
similar way, meaning that individuals gave consistently aberrant or accurate responses 
for the different domains of the measure. All correlations, as indicated by the phi 
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coefficient between each person-fit coefficient, were significant. The correlation 
between the first factor (positive self-esteem) with the second factor (family) was .943 
(p<0.001), the correlation between the first factor and the third factor (negative self- 
esteem) was .137 (p<0.001), and the correlation between the second factor (family) 
and the third factor (negative self-esteem) was .158 (p<0.001). These results offer 
strong evidence that individuals consistently give either accurate or aberrant responses 
to different measures. 


Table 3. Logistic Regression Models Evaluating the Effect of each Cognitive Ability on Person-fit. 
(All subtest) 








Variable B SE Wald y2 OR aaa 
Factor 1 (Positive Self-Esteem) 

Verbal & Reasoning .007 001 24.503* 1.007 .020 
Constant -3.454  .600 33.101* 032 

Factor-2 (Family) 

Verbal & Reasoning .007 001 26.642* 1.007 022 
Constant -3.663  .606 36.510* .026 

Factor-3 (Negative Self-Esteem) 

Verbal & Reasoning 005 001 12.681* 1.005 018 
Constant -2.901  .615 22.228* 055 





Note. B indicate coefficient of logistic regression. SE indicate standard error of estimate. * p<0.001 


4. Discussions 


This study employed a cognitive model to explore how accurately people respond 
to items on self-report assessments. The findings offer useful information to both 
cognitive psychologists and survey methodologists. These results suggest that there is 
a clear relationship between individuals’ cognitive abilities and the person-fit statistics 
for self-report measures. Several points support this finding. First, the items on the 
CSEI contain statements that require individuals to comprehend the meaning of the 
question. To respond to an item, individuals must then match their own responses 
with available responses options on the instrument (in this case, with the degree of 
agreement or disagreement). For example, to respond to the CSEI item “Things 
usually do not bother me,” individuals need to process information about the meaning 
or purpose of the item Individuals then need to understand what “things” means in 
this specific context (i.e., an irritating or upsetting external person or force). 

Second, items on the CSEI are deliberately general, and require individuals to 
contextualize items (Schwarz, 2007). When the item stem contains general 
information, individuals need to interpret the implied underlying context. For 
example, the CSEI item “I cannot be depended on” requires cognitive effort to 
understand. This statement can refer to various contexts, such as performance inside 
the classroom (i.e. completing schoolwork) or outside of the classroom (i.e. 
completing chores). This specific information and clear context is then more easily 
understood than general information. Previous studies have shown that individuals 
infer the pragmatic meaning of item stem in ways following their own reasoning 
(Schwarz, 1996). Interpreting the information involved in an item stem requires 
cognitive abilities, meaning that individuals with cognitive limitations will find it 
more difficult to understand items. 

Third, most of the students who participated in the present study had little 
experience in completing questionnaires, and aberrant responses are commonly given 
by individuals who do not have experience with completing self-reports. According to 
Krosnick (1991), the more experienced respondents are with the survey process or 
familiar they are with the given topic, the more accurate their responses. 

The weak relationship between cognitive abilities and person-fit indices, as 
indicated by low Nagelkerke R-squared values, can be explained by the low 
consistencies among respondents in producing aberrant responses. The correlation 
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among the person-fit statistics for a given individual, for the three factors of CSEI, 
was low. High consistency in giving aberrant responses only occurred between the 
first (positive self-esteem) and second (family) factors of the CSEI (both of which 
involved items in the positive direction), but not between the first and third (negative 
self-esteem; the only factor that involve items in the negative direction) or second and 
third factors. This means that few respondents who gave responses that poorly fit for 
one factor also gave responses that poorly fit for other factors. This low correlation 
may be explained by the fact that only a small amount of the variability in aberrant 
responses can be explained by cognitive abilities. Odd ratios obtained from three 
models based on each subtest of the IST were all of equal values). This low 
correlation among individual person-fit statistics for the three factors of CSEI 
demonstrates that respondents’ capabilities in providing accurate responses depended 
on the specific attribute being measured. 

This finding is consistent with previous studies that have identified correlations 
between psychological ability and traits with person-fit statistics. For example, 
Rudner (1995) failed to detect any relationship between students’ proficiency in areas 
such as counting, measurement, and geometry and person-fit: the correlation between 
person-fit and sub-scale performance was low, ranging from -.433 to 1.616. This 
study also found that few students who gave poorly fitting responses on one measure 
scale tended to give poorly fitting responses on the other scales. Those findings were 
inconsistent with Rudner’s previous findings: in 1983 he had reported that person-fit 
could identify a significant percentage of examinees with abnormal response patterns. 
However, other variables that might moderate the relationship between cognitive 
abilities and response misfit also should be considered, such as motivation or the 
nature of the scale measure (Schmitt, Chan, Sacco, McFarland, & Jennings, 1999). 

Theoretically, aberrant responses will occur more frequently if the self-report 
instrument contains more variations with regard to item location, a parameter by 
which person-fit statistic can detect aberrant responses. For this reason, more variation 
in item location makes it more likely for respondents to produce aberrant responses. 
However, the present study found contradictory results on this point: Although all the 
items on the CSEI are in the same location, many respondents (62% to 67%) were 
identified as giving aberrant responses. By ordering items by their location from 
lowest to highest for a given trait, one can estimate what responses would be 
inappropriate. Aberrant responses are then noted if the respondents get a lower score 
on more difficult items but a higher score on easier items. For this reason, an item 
location distribution with a wide range for each trait is necessary to detect the 
presence of aberrant responses. Since items on the CSEI are distributed over a narrow 
range, false-positive type errors are likely to be more frequent. 

A number of conditions affect good person-fit, including individual ability and item 
parameters, which must be accurately estimated. Individuals must also respond in a 
consistent manner (Rudner, 1995). In the present study those three conditions were 
fulfilled, therefore only a small number of students were identified as giving aberrant 
responses to the scale measure. The students also responded to the scale measure in a 
consistent manner, as evidenced by the fact that correlation between person-fit 
statistics for each student remained low (when there is little variance in response 
distortion, higher correlations between person-fit statistics are difficult to be achieved 
detect). 

There are several limitations to the present study that should be noted. First, using 
person-fit statistics to identify aberrant responses requires that instruments contain at 
least 17 items (as per recommendations by Meijer, Molenaar, & Sijtsma (1994). 
While the instrument used in the present study, the CSEI, has 25 items, these items 
were divided into a number of factors which needed to be analyzed separately. 
Applying this factor component, the number of items within each factor was 6-11 
items fewer that the recommended number. Second, person-fit is dependent on the 
specific characteristics of the study sample, meaning that response pattern identified 
as aberrant in one group but non-aberrant in another group. Response patterns can 
only be identified as aberrant according to a specific range of ability. The third 
limitation of the present study is that only one person-fit statistic (the W-statistic) was 
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employed, even there are many other statistics that can detect person-misfit, and the 
correlation between these different person-fit statistics can be quite low (Karabatsos, 
2003).. In certain measure, a given statistic may be able to detect person-misfit 
properly, but not in other situations. The fourth and final study limitation is that 
alternative Item Response Theory (IRT) models, such as the Graded Response Model 
(GRM) or Rating Scale Model (RSM), both of which fit self-report assessments that 
use Likert-type scales, should be also explored. 

The present study provides strong evidence that scale developers must examine the 
cognitive processes that underlie the measurement process. Scale developers should 
not only pay attention to content when writing the item pool, but should also carefully 
consider what statements will be understood by the intended respondents, as complex 
item stems will only be consistently understood by individuals with high cognitive 
abilities. 
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